Logistic Regression Tree Analysis

This chapter describes a tree-structured extension and generalization of the logistic regression method for fitting models to a binary-valued response variable. The technique overcomes a significant disadvantage of logistic regression, which is interpretability of the model in the face of multicollinearity and Simpson’s paradox. Section 1 summarizes the statistical theory underlying the logistic regression model and the estimation of its parameters. Section 2 reviews two standard approaches to model selection for logistic regression, namely, model deviance relative to its degrees of freedom and the AIC criterion. A dataset on tree damage during a severe thunderstorm is used to compare the approaches and to highlight their weaknesses. A recently published partial one-dimensional model that addresses some of the weaknesses is also reviewed. Section 3 introduces the idea of a logistic regression tree model. The latter consists of a binary tree in which a simple linear logistic regression (i.e., a linear logistic regression using a single predictor variable) is fitted to each leaf node. A split at an intermediate node is characterized by a subset of values taken by a (possibly different) predictor variable. The objective is to partition the dataset into rectangular pieces according to the values of the predictor variables such that a simple linear logistic regression model adequately fits the data in each piece. Because the tree structure and the piecewise models can be presented graphically, the whole model can be easily understood. This is illustrated with the thunderstorm dataset using the LOTUS algorithm. Section 4 describes the basic elements of the LOTUS algorithm, which is based on recursive partitioning and cost-complexity pruning. A key feature of the algorithm is a correction for bias in variable selection at the splits of the tree. Without bias correction, the splits can yield incorrect inferences. Section 5 shows an application of LOTUS to a dataset on automobile crash-tests involving dummies. This dataset is challenging because of its large size, its mix of ordered and unordered variables, and its large number missing values. It also provides a demonstration of Simpson’s paradox. The chapter concludes with some remarks in Section 6.